Expert Systems Without Computers, or Theory and Trust in Artificial Intelligence

Knowledge engineers qualified to build expert systems are currently in short supply. We propose that production of useful and trustworthy expert systems can be significantly increased by pursuing the idea of articulate apprenticeship independent of computer implementations. Making theoretical progress in artificial intelligence should also help. February 14, l984 c © Copyright 1984 by Jon Doyle. I thank Jaime Carbonell, John McDermott, Joseph Schatz, and Derek Sleeman for helpful discussions and comments. This research was supported by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 3597, monitored by the Air Force Avionics Laboratory under Contract F33615-81-K1539. The views and conclusions contained in this document are those of the author, and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the Government of the United States of America. Expert systems and their proponents have caused a revolution in the way we think about work, skill, and their possibilities for automation. This revolution is very important. We now actively seek out tasks for automation that would never have been considered previously. It seems clear that the work of our society and industry includes many economically important (if often mundane) tasks whose automation may be possible with the new techniques. Indeed, this embarrassment of riches has produced a shortage of knowledge engineers trained in constructing expert systems from the current toolkit of knowledge engineering techniques, languages, and systems, so that many worthwhile possibilities go unattended for lack of trained manpower. This bottleneck may not be inevitable, however. The following attempts to clarify the roles that computers and knowledge engineers play in building expert systems, in order to pin down the bottleneck and the possibilities for overcoming it. Our conclusions are that much progress may be possible with articulate human experts and self-conscious human apprentices before one needs to turn to computers or to knowledge engineers, and that the degree to which this may be done depends in part on the level of theoretical understanding in artificial intelligence. If these conclusions are true, the shortage of knowledge engineers may not be as significant as it seems, and might be ameliorated more quickly and effectively by employing readily available human experts and novices to rough out preliminary knowledge bases than by attempting to educate large numbers of knowledge engineers in the current fashion. Articulate Apprenticeship: the essence of knowledge engineering Experience has also taught us that much of [his] knowledge is private to the expert, not because he is unwilling to share publicly how he performs, but because he is unable. He knows more than he is aware of knowing. (Why else is the Ph.D. or the Internship a guild-like apprenticeship to a presumed “master of the craft”? What the masters really know is not written in the textbooks of the masters.) But we have learned that this private knowledge can be uncovered by the careful, painstaking analysis of a second party, or sometimes by the expert himself, operating in the context of a large number of highly specific performance problems. (Feigenbaum, 1977) Although many texts on knowledge engineering stress understanding of data-structures, inference procedures, and skills in manipulating them, as the quoted passage suggests, the key idea in the practice of knowledge engineering is the very old one of apprenticeship. Let us recall how the world of master craftsmen, journeymen, and apprentices worked in the guilds of yesteryear. The master cobbler, say, would take an ignorant apprentice and demonstrate the construction of a shoe, perhaps with a few comments about his actions. The apprentice then attempted to duplicate the feat. But being an ignoramus, having been fascinated by the master’s gold ring instead of by his awl, and having been thrown into a daydream about his girlfriend by the master’s remark about the need for supple hides, he completely botches the intended shoe. The master beats and curses the lout, and demonstrates the other shoe, perhaps making special note of the places where the apprentice made errors. After enough repetitions of these steps, the apprentice becomes a journeyman. At this point he is moderately competent, but more important, has learned something about how to criticize his own work, so that he can improve on his own without requiring the attentions of the master to analyze his errors. If he later gets good enough, he is rewarded with the “assistance” and fees of his own apprentices. Progress has been made since the twelfth century. The most important new twist on this old idea is that of articulate apprenticeship. Instead of relying on largely mute exchanges of performances, we now appreciate the value of masters who try to explain more of what they do, so that the apprentices need not struggle as much trying to perceive what is going on, and apprentices who explain why they did what they did, so that the master can better understand and correct their ignorance and error. In articulate